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Abstract. We have developed a Monte Carlo method to compute the luminosity function of galaxies, based on 
photometric redshifts, which takes into account the non-gaussianity of the probability functions, and the presence 
of degenerate solutions in redshift. In this paper we describe the method and the mock tests performed to 
check its reliability. The NIR luminosity functions and the redshift distributions are determined for near infrared 
subsamples on the HDF-N and HDF-S. The results on the evolution of the NIR LF, the stellar mass function, and 
the luminosity density, are presented and discussed in view of the implications for the galaxy formation models. 
The main results are the lack of substantial evolution of the bright end of the NIR LF and the absence of decline 
of the luminosity density up to a redshift 2 ~ 2, implying that most of the stellar population in massive galaxies 
was already in place at such redshift. 
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1. Introduction 

The study of galaxy formation and evolution implies the 
availability of statistical samples at large look-back times. 
At large redshifts, though, only star forming galaxies will 
be entering the samples obtained in the visible bands 
and, to be able to probe their stellar masses, observa- 
tions at longer wavelengths are needed. In the last decade, 
deep photometric galaxy samples have become available, 
namely through th e obs ervations by HST of the HDF- 
NJWiUiams et al. |1996| ) and HDF-S (Casertano et al. 



200c ), which have been coordinated with complementary 
observations from the ground at near-infrare d (NIR ) wave- 
lengths (Dickinson et al. |2000t da Costa et al. |I99^ ). At the 
same time, reliable photometric redshift techniques have 
been developed, allowing estimates of the distances of faint 
galaxies for which no spectroscopic redshifts can be ob- 
tained nowadays, even with the most powerful telescopes 
(e. g. Con nolly et al. |1997| ; Wa ng et al. |1998| ; Giallo ngo et 

Arnouts et al 



One of the main issues for photometric redshifts is to 
study the evolution of galaxies beyond the spectroscopic 
limits. The relatively high number of objects accessible to 
photometry per redshift bin allows to enlarge the spec- 
troscopic samples towards the faintest magnitudes, thus 
increasing the number of objects accessible to statistical 
studies per redshift bin. Such slicing procedure can be 
adopted to derive, for instance, redshift distributions, lu- 
minosity functions in different bands, or rest-frame colours 
as a function of absolute magnitudes, among the relevant 
quantities to compare with the predictions derived from 
the different models of galaxy formation and evolution. 
This approach has been recently used to infer the star for- 
mation history at high redshift from the UV luminosity 
density, to analyse the stellar population and the evolu- 
tionary properties of distant galaxies (e.g. SubbaRao et 
al 



1996 



Gwyn & Hartwick 1996; Sawicki et al 



al 



Furusawa et al. 



1999 



& Rocca-Volmerange 2002; Bolzonella et al. 2000 and the 
references therein). 



1999 



1998|; Fernandez-Soto et al 

200C| ; Rodighiero et al. 2001|; Le Borgne 



Connolly et al. 1997; Pascarelle et al. 1998; Giallongo et 
al. |1998|; Fernandez-Soto et al. |1999| PoU et al. |2001|), or to 



1997 



derive the evolution of the clustering properties (Arnouts 
et al. |1999|; Magliocchetti & Maddox |1999|, Arnouts et al. 



p002| ) 
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We have developed a method to compute luminosity 
functions (hereafter LFs), based on our public code hy- 
perz to determine photometric redshifts (Bolzonella et al. 
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200C). This original method is a Monte Carlo approach, 
different from the ones proposed by SubbaRao et al. (1996) 
and Dye et al. (2001) in the way of accounting for the non- 
gaussianity of the probability functions, and specially to 
include degenerate solutions in redshift. In this paper we 
present the method and the tests performed on mock cat- 
alogues, and we apply it specifically to derive NIR LFs 
and their evolution on the HDF-N and HDF-S. 

The NIR luminosity is directly linked to the total stel- 
lar mass, and barely affected by the presence of dust ex- 
tinction or starbursts. According to Kauffmann & Chariot 
( |1998D , the NIR LF and its evolution constitute a power- 
ful test to discriminate between the different scenarios of 
galaxy formation, i.e. if galaxies were assembled early, ac- 
cording to a monolithic scenario, or recently from mergers. 
The theoretical NIR LFs derived by Kauffmann & Chariot 
exhibit a sharp difference between the two models at red- 
shifts z > 1. The PLE models foresee a constant bright- 
end for the LF, whereas hierarchical models are expected 
to undergo a shift towards faint magnitudes with increas- 
ing redshift. In this paper we compare the theoretical pre- 
dictions with the observations on the HDFs, in order to 
extend the analysi s perf ormed from spe ctros copic surveys 
(Glaz ebrook et al. 1995 , Songaila et al. 1994 , Cowie et al. 
199£) up to z 2. The comparison between the present 



Table 1. Characteristics of photometric data used in this 
paper: effective wavelength, bandpass and AB conversion 
(convAB = 'm-AB — TfiYcga.) for the different filters, and 
limiting magnitudes in the HDF-N and HDF-S (see text). 



NIR LF results and a similar study in the optical-UV 
bands, obtained with the same photometric redshift ap- 
proach (Bolzonella et al. Paper II, in preparation), could 
provide with new insights on the galaxy formation sce- 
nario. 

The plan of this paper is the following. Section || 
gives a brief description of the photometric catalogues 
used to select the samples, their properties being dis- 
cussed in Sect. |[ In Sect. § we describe the technique 
conceived to compute the luminosity functions using the 
hyperz photometric redshift outputs, and the test of the 
method through mock catalogues. The results obtained on 
the HDFs near-infrared LF are presented and discussed 
in Section ^ together with the Luminosity Density and 
the Mass Function derived from them. The implications 
of the present results on the galaxy formation models are 
discussed in Sect. |6| and the main results are summarized 
in Sect. 0. 

Throughout this paper we adopt the cosmological pa- 
rameters fio = li — 0, when not differently speci- 
fied. Magnitudes are given in the AB system (Oke 1974). 
Throughout the paper, the Hubble Space Telescope fil- 
ters F300W, F450W, F606W and F814W are named C/300, 
^450, Veoe and /su respectively. 



2. Photometric catalogues 



The HDF- N (WiUiams et al. |T99§) and HDF-S (Casertano 
et al. 2000) are the best data sets to which the photometric 
redshift techniques can be applied, because of the wave- 
length coverage extending from the U^qq to the Kg bands 
through a combination of space and ground based data. 
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25.0 
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1.871 


25.0 


25.3 



and of the accurate photometry available. Table |l| gives 
the characteristics of the filters involved. 

For the HDF-N we adopted the photometric catalogue 
provided by Fernandez-Soto et al. (1999). These authors 
used the SExtractor (Berlin & 
and detected objects in the /gi4 



Arnouts 1996) package 
image. The final cat- 
alogue consists of a total of 1067 galaxies (after exclu- 
sion of known stars) in 5.31arcmin^. The reference aper- 
ture used to measure the fluxes in the different bands 
is defined by the threshold isophote on the /814 image. 
This catalogue is divided in 2 zones: the deepest one 
(Zl) consists of 946 objects in 3.92 arcmin^ and it has 
been limited to /8i4(^i?) < 28 (in "best" magnitudes, see 
SExtractor user manual), the shallowest one (Z2, includ- 
ing field edges and the planetary camera) contains 121 
objects to Ig,i4{AB) < 26, in an area of 1.39 arcmin^. 

For the HDF-S wc adopte d the photometric catalogue 
prepared by Vanzella et al. |200l| and available on the 



web|^. The catalogue contains 1611 objects, detected in 
the Vgoe + -^8 14 image and with magnitudes measured in- 
side a variable aperture suited to compute photometric 
redshifts. The NIR imaging has been carried out with the 
near-infrared spectrometer/imager VLT-ISAAC and the 
photometry has been measured taking into account the 
different PSF of NIR images. 

3. Sample properties 

3.1. Photometric Redshifts 

To compute photometric redshifts, we converted the fluxes 
fu into AB magnitudes, assigning the magnitude value of 
99 (corresponding to an undetected object in our photo- 
metric redshift scheme) in case of negative fluxes, negative 
error fluxes, or S/N = /^/A/j, < 1. Moreover, we added 
in quadrature a photometric error of 0.02 magnitudes in 
all filters, to account for systematics in the zeropoints. 

Then, we applied the public code hyperz^ to compute 
the photometric redshifts of galaxies in the HDF-N and 
HDF-S. A full description of the technique and the anal- 
ysis of the performances on the HDFs can be found in 

^ http:/ /www. stecf.org/hstprogrammes/ISAAC/HDFSdata. html 
^ http://webast.obs-mip.fr/hyperz/ and mirror sites 
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Fig. 1. Absolute magnitude in the _B-band as a function of photometric redshift in the HDF-N and HDF-S for objects 
with -6450 < 28.5. The filled dots represent the subsample with Kg < 24, used to estimate the luminosity functions 
in Section ^. The solid lines mark the limiting absolute magnitude computed using the apparent limiting magnitude 
of the sample in the -6450 band, and the mean fc-correction over a mix of spectral types. The dashed lines display the 
same result when using the limiting magnitudes for the NIR filters. 



Bolzonella et al. (2000). Here we only recall that an inter- 
esting characteristic of the hyperz code is the possibility 
of computing probability functions in the redshift space. 
We will make full use of this facility to derive the LFs. 

A full description of the different parameters used by 
hyperz can be found in the User's Manual of the code, 
available on the web pages. The most relevant parameters 
used here are the following: 

- The limiting magnitudes applied in case of undetected 
objects are given in Table |l|. We choose P(< to) = 0.8 
in the cumulative histograms to set the adopted limiting 
magnitudes, which roughly correspond to objects with a 
S/N - 1. 

- We used Calzetti's ( ^000 ) law to produce reddened 
templates, with Ay ranging from to 1.2 magnitudes. 

- The Lyman forest blanketing is modelled according 
to the standard prescriptions of Madau ( |1995D , without 
variations along the different lines of sight. 

- As galaxy template set we selected the 5 "standard" 
SEDs bu ilt us ing the GISSEL98 library by Bruzual & 
Chariot (|l993|) , with solar metallicity; the CWW SEDs 
were not considered because they are redundant. 

- The range of absolute magnitudes Mb constraining the 
allowed solutions is [—28,-9]. 

A comparison between the photometric and spectro- 
scopic redshifts has been shown in Bolzonella et al. ( 20001 ). 
Here we assume that galaxies in the photometric sam- 
ple will follow the same behaviour, i.e. the same disper- 
sion around the true values. Beyond /814 = 25 our spec- 
troscopic knowledge falls dramatically. Very few objects 
are available to calibrate the photometric redshifts in this 
domain, and so larger discrepancies are always possible. 
Nonetheless, these objects will remain beyond the spec- 
troscopic limits till the arrival of larger telescopes in the 



future, and photometric redshifts are presently the only 
available tool to study them. 

We used the results of the photometric redshift esti- 
mate to examine some properties of the sample and to 
give an estimate of the evolution in the samples. 

Figure displays the absolute magnitude in the B- 
band, obtained as standard output of hyperz, as a function 
of the photometric redshift for objects with -6450 < 28.5. 
The thick solid line represents the limiting absolute mag- 
nitude computed for a B = 28.5 object and a mean k- 
correction computed over a mix of spectral types, with 
the method explained below in Sect. [4.3[ For comparison, 
the thick dashed lines show the same result when using the 
limiting magnitudes given in Table ^ for the NIR filters. 
It is worth to remark the sequence of theoretical absolute 
magnitudes, in agreement with the observational limits, 
obtained without imposing stringent constraints in Mb- 

In Fig. H we plot the observed /814 magnitudes versus 
the photometric redshift, viz the Hubble diagram. The 
mean redshift per magnitude bin is also shown, with its 
la dispersion. The faintest objects are also the farthest 
ones, as we expect for an expanding universe. The posi- 
tion on this Hubble diagram of a local L* galaxy is also 
shown for comparison, for the two sets of cosmological 
parameters used in this paper. In the HDF-S there are 
some objects well above the L* lines: most of them are 
objects with P(2;phot) = 0, shown as triangles. Nearly half 
of these objects are classified by SExtractor as stars in the 



Vanzella et al. (2001) catalogue. When we fit their SEDs 
with stellar templates (Pickles 1998) and quasar spectra 
(according to the method described by Hatziminaoglou et 
al. 2000), only one of these objects has colors fully con- 



sistent with a highly reddened star. In all the other cases, 
these objects are not well fitted neither with the standard 
SEDs for galaxies nor with the stellar or quasar templates. 
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Fig. 2. Hubble diagram for objects in the HDF-N and HDF-S samples. Filled symbols represent the subsample with 
Ks < 24. Triangles show the objects for which ^(zphot) = everywhere. Dashed and solid lines display the location 
on this diagram of a local L* galaxy, with cosmological parameters (Oq = 1, f^A — 0) and (fio = 0-3, = 0.7) 
respectively, with Hq = 50kms^^ Mpc^^. The vertical dotted line marks the Isi4 — 28.5 magnitude at which the 
signal-to-noise ratio is S/N = 3. The mean redshift and the la dispersion have also been plotted. 




Fig. 3. Colour /814 — Ks as a function of photometric redshift, compared to the predictions for different synthetic 
SEDs. From top to bottom: Elliptical galaxy of fixed 10 and 5 Gyr age, evolving E, Sa, Im, and Im of fixed 1 and 
0.1 Gyr age. Empty dots illustrate the sample with Kg < 26 in the HDF-N and HDF-S samples, corresponding to 
S/N = 1, whereas filled dots represent objects with Kg < 24, i.e. with S/N > 3. The observed colours are enclosed in 
the limits of colours foreseen from the synthetic SEDs. 



Thus, we have no reason to exclude them when computing 
LFs. 

The inspection of these two figures indicates that the 
statistical properties of the photometric redshift samples 
display the expected behaviour. According to our previ- 
ous simulations, aiming to reproduce the properties of the 
HDFs (Bolzonella ct al. |2000D, the redshift distribution 



beyond the spectroscopic limits can be considered as reli- 
able. 

In Fig. ^we plot the apparent colour /814 — Ks versus 
the photometric redshift, as well as the colours computed 
from the synthetic SEDs of different types: from top to 
bottom, an Elliptical galaxy at fixed ages of 10 Gyr and 



5 Gyr, evolving E, Sa, Im, Im of fixed 1 Gyr and 0.1 Gyr 
age. A few objects redder than old ellipticals seem to be 
present at z < 1 in both fields. Also, a group of objects 
with /814 — Ks > 2.6 and 1 ^ z ^ 2 is detected: these ob- 
jects are EROs according to the usual criteria (e.g. Cimatti 
et al. |1999|; Scodeggio & Silva ^000|; Moriondo et al. pOOO). 



3.2. Redshift distribution 

Figure |4| shows the redshift distributions, as obtained from 
the photometric redshift computation, for the HDFs cata- 
logues. We have compared our result on the HDF-N with 
the redshift distributions obtained by Fernandez-Soto et 
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Fig. 4. Rcdshift distributions obtained for the Isu selected samples in the HDF-N and HDF-S by different authors. 
Left panel: N{z) in the HDF-N obtained using the SUNY catalogue by hyperz, hyperz with a Monte Carlo method, 
Fernandez-Soto et al. (1999) and Fontana et al. ( 2000| ), with limiting magnitudes of /814 — 28.5 (dotted line) and 
/814 — 26 (solid line). Right panel: N{z) in the HDF-S, with the same two limiting magnitudes. Fontana et al. (2000) 
used the SUNY catalogue, limiting their analysis to /814 = 27.5 (dashed line). 




Fig. 5. Redshift distributions obtained for the Kg selected subsamples in the HDF-N and HDF-S by different authors. 
Left panel: N{z) in the HDF-N obtained using the SUNY catalogue by hyperz, hyperz with a Monte Carlo method, 
Fernandez-Soto et al. ( 1999| ) and Fontana et al. ( 2000| ) with two limiting magnitudes: Kg < 25 (dotted line) and 
Ks < 23.5 (solid line). Right panel: the same in the HDF-S. In the lower right panel the solid line represents the N{z) 
at Kg < 23.5 obtained by Fontana et al. ( ^000 ) and the dotted line the N{z) at the same limit by Rudnick et al. 
(^00 ll). 



al. (|1999| , SUNY group) and Fontana et al. mm Roma 
group) , using the same catalog (see Section In the 
HDF-S, we compared our redshift distribution, obtained 



with the catalogue by Vanzella et al. ( [2001 ) with the N{z) 
comput ed usi ng the SUNY catalog ue by Fernandez-Soto 
et al. ( |1999D and Fontana et al. ( ^OOO] ). We have also 
computed a median redshift distribution, obtained with 
the Monte Carlo method we applied to estimate the lumi- 



nosity function, and detailed in Section 4.2. This redshift 
distribution takes into account the probability functions 
of individual objects, in such a way that objects will be 
scattered in their allowed range of photometric redshifts. 



according to their P(zphot)- Objects with flat ^(zphot) will 
spread over a wide range of Zphot, whereas objects with 
narrow ^(zphot) will be always assigned to a redshift close 
to the best fit one. We show the result of 100 iterations, 
represented by the median and the error bars, computed 
as 10% and 90% of the values distribution in each redshift 
bin. 

Considering the fainter sample of the HDF-N pre- 
sented in the left panel of Figure ^, the Kolmogorov- 
Smirnov test shows that the hyperz and SUNY distri- 
butions are compatible, whereas the hyperz and Roma 
distributions are not. We reject the hypothesis that two 
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distributions were issued from the same parent distribu- 
tions when the significance level is lower than a conser- 
vative value of 1%. Using the same criterium, the median 
N{z) computed with the hyperz-Monte Carlo method is 
compatible with both the SUNY and the Roma distribu- 
tions. At /814 < 26, the KS test shows that each N{z) is 
consistent with the others. 

In the HDF-S, we show the comparison of N{z) for the 
same two limiting magnitudes, Isu < 28.5 and /814 < 26. 
Fontana et al. ( ^OOOD selected their subsample imposing 
-^814 ^ 27.5 (dashed line). The KS test for the subsam- 
ple with /814 < 28.5 shows that the distributions are not 
drawn from the same parent distribution, but considering 
the bright subsample with /814 < 26 the distributions are 
fully consistent each other, even if they are obtained from 
different photometric catalogues. 

In Figure |5| we show the same comparison carried 
out for the A'^-band selected subsample we used in the 
following of this paper. We have analysed the distribu- 
tions obtained when selecting objects with < 25 and 
Kg < 23.5. In the HDF-N the agreement among the dif- 
ferent N{z) distributions is remarkable: by means of the 
Kolmogorov-Smirnov test we obtain that we can reject the 
possibility that the distributions were drawn from differ- 
ent parent distributions with a confidence level ranging 
from 23% (minimum) and 70% (maximum) at < 25, 
and a probability ranging from 41% to 100% at Kg < 23.5. 

In the HDF-S the same comparison shows that the 
hyperz and SUNY distributions are not compatible, even 
choosing the conservative value of 1% for the confidence 
level, whereas the hyperz and Roma results are marginally 
consistent. On the other hand, the redshift distributions 
of the bright subsample show a better agreement, with 
probabilities in each case larger than the chosen limit, 
even if the catalogues are different, in particular in the 
NIR dataset. In the lower right panel we also show the 
redshift distribution obtained by Rudnick et al. ( 2001 ) for 
a sample limited to Kg < 23.5 and a catalogue built by 
these authors using ISAAC- VLT images. Also in this case 
we found that the redshift distributions are compatible. 
Even considering the subsample with Kg < 24, i.e. the 
limit chosen computing LFs, the redshift distributions are 
in general consistent. 

In conclusion, some small differences are observed in 
the redshift distribution when using different methods, 
mostly affecting the faintest samples, whereas the results 
are compatible for the bright ones. At faint limits in mag- 
nitude, the redshift distribution obtained with the hyperz- 
Monte Carlo method avoids introducing spurious features, 
and it makes the relevant N{z) distributions compatible. 
At brighter limits, by means of the comparison with the 
other photometric redshift estimates, we have shown that 
the results are consistent and characterized by a high qual- 
ity in the redshift determination. The differences among 
different authors become very smooth when the Monte 
Carlo approach is used; this procedure allow us to com- 
pute reliable LFs in Sect. |[ 



3.3. Cumulative redshift distribution 



Kauffmann & Chariot (199S) argued that the cumulative 
redshift distribution in the K-hand can be used as a test 
for the scenarios of galaxy formation. In Fig. ^ we have 
compared our data with the theoretical expectations given 
by Kauffmann & Chariot (1995), in the case of mono- 
lithic (PLE) and hierarchical galaxy formation scenarios. 
The original K-hand magnitude bins have been shifted by 
^ 2 magnitudes to match the AB magnitudes used here. 
The fraction of galaxies at high redshifts is much larger 
in the case of the PLE models than in the hierarchical 
scenario. The observed HDFs cumulative redshift distri- 
bution lie between the two models and, in any case, it 
is always below the predictions of the PLE models. This 
r esult was already pointed out by Kauffmann & Chariot 
( 1998 ) , when comparing their predictions with the results 
f ound with the Songaila et al. (1994) and Cowie et al. 
( 1996| ) samples, in the two brightest magnitude bins. The 
present results on the HDFs extend this trend towards 
the faintest magnitude bins. However, this results must 
be considered with caution, mainly because of the small 
size of the surveyed fields and uncertainties affecting the 
models. For instance, Kauffmann & Chariot derived their 
PLE model from the S-band LF. The expectations de- 
rived from monolithic and hierarchical scenarios should 
be more compatible in the forthcoming new generation of 
models (Pozzetti, private communication). On the other 
hand, expectations derived by Fontana et al. ( 1999| ) for a 
hierarchical model in the framework of a ACDM cosmol- 
ogy seem to be in good agreement with the HDFs data. 
Due to the models uncertainties affecting the comparison 
of cumulative redshift distributions, we applied the more 
powerful test on the i^-band LF proposed by Kauffmann 
& Chariot (1998), whose results are shown in Sect. |^. 



4. Luminosity Functions 

The Luminosity Function represents the number of objects 
per unit volume with luminosities in the range [L, L + 
dL]. Differently from galaxy counts, distances are involved 
in the LF computation, where the intrinsic luminosities 
are considered rather then the apparent ones (except the 
LFs of objects belonging to a unique structure, like galaxy 
clusters). This characteristic makes the LF an important 
cosmological test, containing much more information than 
galaxy counts. 

The LFs are of crucial importance in the description 
of sample statistical properties in observational cosmol- 
ogy. The LF can measure the amount of luminous matter 
in the universe, depending on the cosmological parame- 
ters. Moreover, the analysis of its characteristics and its 
evolution provides fundamental insights on the galaxy evo- 
lution mechanisms and can constrain the formation epoch. 
Studying the LFs in a wide range of absolute magnitudes 
is a necessity to understand the galaxy formation process. 
To attain this goal, more and more fainter apparent mag- 
nitudes must be reached. 
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Fig. 6. Cumulative redshift distribution in the Kg band selected samples in HDF-N and HDF-S obtained with the 
Monte Carlo method (thick solid lines), compared to the theoretical expectations by Kauffmann & Chariot (1998): 
solid and dotted lines correspond to fio = 1 and SIq = 0.2 PLE models with null cosmological constant, whereas 
dashed lines correspond to their hierarchical galaxy formation model based on a fio = 1 cosmology. The 3 apparent 
Ks magnitude bins roughly correspond to the original ones used by Kauffmann & Chariot ( 199§| ). The light gray 
cumulative histogram in the lower panels represents N{> z) for the sample with < 23, whereas the light gray 
continuous line is the theoretical expectation at the same limiting magnitude computed by Fontana et al. (1999) for 
a hierarchical model based on a ACDM cosmology. 



4.1. LF estimators 

We applied three different methods to estimate the L F: the 
1/Vmax, the C- and the S TY methods. Willmer ( |l997|) 
and Takeuchi et al. ( 2000 ) have recently reviewed and 
compared these estimators. 

The l/14nax is the so-called classical method^ first pub- 
lished by Schmidt ( 196^ ), and detailed later by Felten 
( 119761 ). It was conceived for quasars, as many of the other 
LF estimators, but it is extensively applied in the galaxy 
LF computation: 



max(2i ,2n 



1(2:2, 



°) dV_ 
I dz 



dz 



(1) 



where Vmax is the minimum comoving volume in the sur- 
vey in which the galaxy i with absolute magnitude 
remains observable, given zi and Z2, the redshift range of 
the survey. It is a non-parametric method, i.e. it does not 
assume a shape for the LF. One of the advantages of this 
method is its simplicity. Moreover, it is a non-parametric 
estimator and thus it gives the shape and the normaliza- 
tion at the same time. Also, it is an unbiased estimator of 
the source density. Of course, there are also some short- 
comings. First of all, the 1/Knax estimator is very sensitive 
to density fluctuations, especially in pencil beam surveys. 
Furthermore, one loses information on where in the mag- 
nitude bin a galaxy is located, but for small bins (c?M) it 
is a reasonable estimate. 

The C~ method was introduced by Lynden-Bell 



(1971); the original technique was simplified and devel- 
oped by Cholonicwski (1987) in such a way as to compute 
simultaneously the shape of the LF and the density. We 



adopted the Choloniewski approach: the observed distri- 
bution of galaxies is assumed to be separable in its depen- 
dences on the absolute magnitude M and the redshift z. 
As the 1/Knax estimator, this method is non parametric, 
but it has the advantage of being insensitive to density in- 
homogeneities. SubbaRao et al. ( |1996| ) realized a modified 
version of this method, to take into account a continuum 
distribution of redshifts arising from the photometric red- 
shift computation. We discuss this case later. 

We also applied the STY method proposed by 



Sandage, Tammann & Yahil (197£). This estimator uses a 
maximum likelihood technique to find the most probable 
parameters of an analytical LF (f>{M), in general assumed 
to be the Schechter function: 



(f>{L) dL = (/)* 



exp 



L 

L* 



(2) 



that, transformed into magnitudes, becomes: 

4>{M)dM = 0.41n(10)(/)*10-°-'*(*^-^^*'("+i) x 

exphlO-^-^^^^-^^'^ldM. (3) 

Because the data are not binned, the method takes ad- 
vantage of all the information in the sample. Imposing a 
functional shape, we cannot test if the assumed (j){M) is 
a good representation of data. Then the non-parametric 
methods allow to plot the data points and the STY esti- 
mator can be used to search for the better parametriza- 
tion. The unknown parameters of equations ^ and ^ are 
the normalization 0*, the faint end slope a, that assumes 
negative values, and the characteristic luminosity L* or 
magnitude M* , that marks the separation between the 
exponential law prevailing in the bright part of the LF 
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and the power law with index a dominant in the faint 
end. All these parameters can depend on the morphologi- 
cal type of galaxies under consideration. Several methods 
have been conceived to compute in an unbiased way the 
mean galaxy density and thus the parameter 0*. A de- 
tailed discussion of the density estimators can be found in 



Davis & Huchra ( |1982D or in Willmer (|1997|). 

Davis & Huchra (1982) derived a minimum variance 
estimator: 



J2f=l NjjZr) wjZj) 

Qs{z)w{z)%dz 



(4) 



where Ni is the number of galaxies &i z = Zi (in general 
Ni = 1, but it can take different values if galaxies are 
divided in redshift bins), whereas w{zi) are the weights. 
The estimator called by Davis & Huchra ( 1982 ) can be 
derived from equation^ by taking w{zi) = 1: 

"3 rZ2 , . dV , ■ 



i:: 



dz 



Here all the observed galaxies are equally weighed and 
then the estimator is quite stable, but it is affected by 
large scale inhomogeneities: if (t^/Kiax) > 0.5, the de- 
duced value of 713 can be overestimated. 

4.2. Beyond the spectroscopic limit: the Monte Carlo 
approach 

One of the problems in the study of LFs is the availability 
of extended samples, in terms of the number of galaxies 
involved in such samples, in the range of magnitudes at- 
tained (to study in detail the faint-end behaviour) and 
in large redshift domains (to study the evolution beyond 
z ~ 1). However, such a sample is not easy to acquire in 
spectroscopic surveys, and has not yet been obtained. 

A viable alternative solution is the use of the deeper 
and faster photometric surveys, which allow to use pho- 
tometric redshifts instead of spectroscopic ones. A spec- 
troscopic subsample is anyhow recommended, to calibrate 
and check photometric redshifts. A suitable survey ac- 
cording to these requirements is, once more, the HDFs. 
Different groups put a particular effort in the attempt 
to compute the LF of HDF-N (Gwyn & Hartwick 



Sawicki et al. 1997; Mobasher et al. 1996: Takeuchi et al 



1996 



200q ). 

Up to now, the approaches used to compute the 
Luminosity Functions by means of the photometric red- 
shift technique, in the HDF-N or other fields, can be sum- 
marized as follows: 

— the photometric redshift is assumed to be the true 
redshift o f the galaxy, i.e. Zp hot = -^truc (Gwyn & 
Hartwick |1996|: Sawicki et al. |1997|; Mobasher et al. 



1996; Takeuchi et al. 2000); 



for each object, a smooth distribution of redshifts 
around Zphot is considered, with a Gaussian probabil- 
ity function P{z). The rms of the Gaussian distribu- 
tion can be either a fixed value or a value depending 



on the apparent magnitude of the galaxy under con- 
sideration. In this case, the rms is derived from the 
expectations obtained from the analysis of the spec- 
troscopic subsample. The smoothed redshift can be 
used in different ways: SubbaRao et al. (1996) con- 
ceived a modified version of the C~ method to account 
for a continuous distribution of redshifts around Zphot • 
Liu et al. (1998) obtained an estimate of the LF using 
the 1/Vinax method and attributing a series of frac- 
tional contributions to the luminosity distribution in 
the redshift space around Zphot- Dye et al. 2001 used 



the Gaussian distribution to realize a series of Monte 
Carlo iterations, where the value of each Zphot is chosen 
randomly according to the Gaussian distribution. 

These approaches are subject to some shortcomings, 
due to the nature of the photometric redshift technique 
itself. First, the photometric redshift is only an esti- 
mate of the true redshift, and then we can only suppose 
ZpYiot ~ ztruc- Second, in most cases the probability distri- 
bution as a function of redshift, P{z), cannot be assimi- 
lated to a Gaussian function, because degeneracies and un- 
certainties produce complex distribution functions, often 
with the presence of several distinct peaks. These features 
of the photometric redshift estimate lead to different dis- 
tances di, absolute magnitudes Af, volumes Knax and also 
to different fc-corrections, because in general the spectral 
type of the best fit at a fixed redshift can be different from 
the best fit type of the surrounding redshift steps. Then, 
all these details about the photometric redshift technique 
must be taken into account when computing the lumi- 
nosity functions of galaxies belonging to a photometric 
survey. 

To compute the LFs in the HDF-N and HDF-S, we 
adopt ed a M onte Carlo approach, different from the Dye 
et al. (2001) method in the way of accounting for the non- 
gaussianity of the photometric redshift errors. Specifically, 
to assign the photometric redshift used in each iteration of 
the LF estimate, we build the cumulative function Pcum(-z) 
from the P{z), as described in Fig. During each Monte 
Carlo iteration, for every galaxy we randomly select a 
number between and 1, corresponding to a value of the 
redshift Zprob that we assign to the considered galaxy. The 
probability of obtaining a given value of Zpiob is related 
to the P{z) distribution: when the Pcnra{z) remains hori- 
zontal, it means that the probability relative to those 
redshifts is near to 0, and then it is almost impossible 
to select them by choosing a number in the interval [0, 1]. 
Viceversa, vertical regions of the Pcum(-z) curve correspond 
to very likely values of the photometric redshift. 

Proceeding in this way, we use all the informations con- 
tained in the . log_phot file produced as output by hyperz, 
we take into account the existence of multiple solutions, 
and we are able to compute the correct fc-corrections, 
knowing all the characteristics of the best fit SED at 
z = Zpj-oi^, i.e. the associated spectral type, age and Ay- 

The final data points of the LF or its fitting parame- 
ters will be evaluated by means of the median over many 
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Fig. 7. An example of the procedure to select the photometric redshift randomly and according to the probability 
distribution of the object. Left: The function P{z) as computed by hyperz, renormalized to the maximum value. Right: 
The corresponding cumulative probability function. A random number between and 1 is drawn. The value of Zprob is 
computed by linear interpolation between the two redshift steps bracketing the value of the cumulative function equal 
to the random number. 



Monte Carlo realizations. The errors will be immediately 
found after sorting, by locating the values Xk whose in- 
dexes k correspond to the assigned probability. In this 
way we can compute the confidence intervals at different 
levels. 

Another possibility to take into account the uncertain- 
ties and the information contained in the photometric red- 
shift procedure, is the method described by Arnouts et al. 
( |1999| ): they performed Monte Carlo simulations to test 
the effect of the photometric errors on redshift estimates. 
They assigned a random magnitude according to the pho- 
tometric rms and verified if the changes of redshift could 
affect the statistics inside the redshift slices they used to 
divide the sample. We chose not to follow this approach, 
because the degeneracy among different parameters can 
lead to a smoothed P{z), even when the photometric er- 
rors are small. In the procedure followed by Arnouts et al. 
1999| ), the presence of secondary and significant peaks in 



P{z) is not taken into account. 

In the following, we adopt a cosmological model with 
parameters fio = 1, Ha = to facilitate the comparison 
of our LF results with other surveys. We compute the 
LFs also for the fashionable cosmological model flo = 0.3, 
Qa ^ 0.7. 

4.3. k-corrections 

Usually, fc-corrections in redshift surveys are computed 
from spectra at z = 0, after attributing a spectro- 
morphological type to each object (e.g. Lilly et al. 1995 ; 
Loveday et al. 1999). When it is impossible to separate 
galaxy types, a statistical fc-correction can be computed 
considering a mix of morphological types (e.g. Zucca et 
al. 1997). The SED fitting technique used to determine 



correction for each object, obtained directly from the best 
fit SED. 

The absolute magnitude of an object at redshift z, in 
a given filter M\ is: 



Mx = mx - Slogdi - 25 - kx{z) 



(6) 



where c?l is the luminosity distance in Mpc, mx is the ap- 
parent magnitude, and kx{z), the /c-correction, is defined 
by: 



kx{z) = Mx{z,tQ) - Mx{0,to) 

JfiX,to)R{X) dX 



= 2.5 log 



Jf{j^,to)RiX)dX 



+ 2.51og(l + z). (7) 



photometric redshifts allows to compute a well suited k- 



Here to is the time corresponding to the present epoch 
(z = 0); therefore, the /c-correction is a pure geometrical 
correction which transforms the observed magnitude into 
the magnitude in the rest-frame of the observed galaxy, 
assuming no spectral evolution. This approach is straight- 
forward at low redshifts, where the differences induced by 
spectral evolution are negligible, and then the empirical 
SED f{X,to) of a galaxy at to {z ^ 0) can be used to 
derive the fc-correction of a galaxy seen at redshift z. For 
high redshift galaxies, the uncertainties on the evolution 
of the SEDs from /(A, to) to /(A, t^), that is the evolution- 
ary correction, make the previous approach not reliable in 
the context of this paper. The most straightforward way 
to compute the fc-correction is to apply equation ^ on the 
best fit SED at redshift z. 

In practice, to minimize the assumption on the best 
fit SED, we choose the apparent magnitude in the filter i 
which is closest to the rest-frame filter selected for the LF, 
that we call k, and we compute the absolute magnitude 
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Fig. 8. fc-corrections in J and Kg bands, for the same spec- 
tral types contained in the hyperz package. Lines from top 
to bottom represent fc-correction from early to late type 
galaxies. Left: fc-correction in magnitudes, computed from 
the different to SEDs. Right: fc-correction in magnitudes, 
computed from the evolving SEDs. 



in the AB photometric system through the equation: 



Mk 



1, - 2.5 log 
-2.5 log 



//sed(tti) R^WdX 

J /vcga(A)i?fc(A) dX 
//vcga(A)i?,(A)dA 

//sED(A)i?fe(A) dX{l + z) 



-5\ogdL - 25 - 2.5 log ^ 



//SED (iTl) Rk{X) 



-convAB/ 



convAB,fc 



dX 

(8) 



where the first two lines give the correction to obtain the 
apparent magnitude in the filter fc, the third line corre- 
sponds to the passage from apparent to absolute magni- 
tudes and the forth line contains the corrections to trans- 
form the input AB magnitudes in the Vega system for the 
i filter and viceversa for the fc filter, to obtain the final 
AB magnitude. 

Thus, we do not introduce the evolutionary correction 
explicitly, but we use the most reliable fc-corrected mag- 
nitudes, based on the best fit SEDs. In this way, the evo- 
lution of the galaxy population can be directly compared 
with the expectations from the different models of galaxy 
formation and evolution. 

One of the advantages of working on NIR wavebands 
is that the fc-corrections are small and nearly independent 
on spectral type, thus minimizing the uncertainties in the 
estimate of the absolute magnitudes. In particular, shallow 
redshift surveys use — 2.5z for all galaxy types in the K 



band (Loveday 2000; Glazebrook et al. 1995). This repre- 
sents a good approximation for galaxies with z < 0.30, but 
not in our case, because we are dealing with higher redshift 
objects. Therefore, we adopted a consistent technique, us- 
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Fig. 9. /814 — Ks CM diagrams for the ii'^-band se- 
lected subsamples. Thick solid lines illustrate the limiting 
colours computed according to Table |l|, corresponding to 
80% of the cumulative histogram of apparent magnitudes, 
whereas thick dotted lines correspond to the very faint 
limit of magnitudes in the whole catalogue. The dashed 
lines represent the actual limits used to compute the LFs. 



ing the SED corresponding to the best fit at the selected 
^phot from the Monte Carlo procedure. In Fig. ^ we show 
the fc-corrections in the J and Ks bands used in the LF 
computation, for both the usual correction based on the to 
SED, and the fcovoi-correction, computed from the evolved 
SED at tz, assuming that galaxies form at z = 10. The 
lines represent, from top to bottom, the fc-corrections from 
early to late type galaxies. The fcovoi-correction may not 
represent the correction actually applied, because galaxies 
can have best fit SEDs with younger ages than t^, but the 
plot is valid to show the overall trend. 

In particular for the i^^-band, we can remark that up 
to redshift z = 3 the fc and fcevoi-corrections are nearly 
independent on the spectral type and small. For these rea- 
sons we considered reliable the extrapolation of absolute 
magnitudes up to at least redshift ^ 2 in the filter and, 
with some caution, up to z 3. 

4.4. Incompleteness 

The advantage of using a photometric catalogue is that it 
is less subject to incompleteness then spectroscopic red- 
shift surveys. The incompleteness in redshift surveys, that 
can affect the LF estimate, can be not only magnitude- 
dependent, but it can also arise as a function of galaxy 
type or redshift, and in some cases it is impossible to take 
it into account. 
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Obviously, also photometric surveys are affected by in- 
completeness. The SUNY catalogue in the HDF-N is, by 
construction, an 1^x4 band selected sample (Fernandez- 
Soto et al. |1999| ), whereas objects in the ISAAC HDF-S 
catalog are detected on the V606 + -^8i4 image. Even though 
we use a limited sample in the subsequent calculations, 
it is worth to check on the possible color-selection effects 
which could affect the two fields in different ways. Figures 
^ and ^ display the color-redshift and color-magnitude di- 
agrams. As shown in Fig. ||, very red objects in Isu — Kg, 
with faint Kg > 24.5 magnitudes, could be missed due to 
selection criteria. To avoid such effects of colour selection, 
we adopted a limiting magnitude Kg = 24, corresponding 
to S/N ~ 3. Moreover, this selection allows us to use the 
whole area of 5.31 arcmin^ in the HDF-N, combination of 
the Zl and Z2 zones, because at this limit the colour dis- 
tributions of objects with redshifts between and 2 have 
a similar average: {Isi4 — Ks) = 1.28 in Zl and 0.95 in 
Z2, with a large dispersion in both zones. In the Z2 zone, 
even for the faintest objects in Ks, the colours up to 2 - 
2.5 are allowed: in the Zl zone most objects have colours 
below this value, thus we do not introduce any bias by 
combining the data belonging to the two zones. 

At the selected limit in magnitude, blue galaxies have 
about the same chance to be observed than the reddest 
ones in the ifg-band selected subsamples in both fields. 

Another type of incompleteness could arise from the 
surface brightness effect: when objects are detected at 
bright surface brightness limit, then the LF estimate 
could be affected, with M* becoming fainter, 0* smaller 
and a slightly flatter (Cross & Driver 2002 and refer- 
ences therein). In our case, the detection up to faint 
surface brightness used by the authors of the catalogues 
{li\im{h^^) - 26arcsec-2 in both the HDF-N and HDF- 
S) will not induce significant effects on the LF estimates. 
In particular, the bright end of the LF, on which we base 
our conclusions, will not suffer strongly from the men- 
tioned effect. The same inference can be demonstrated for 
other types of incompleteness, such as the detection and 
measurement algorithm, or the cosmol ogical dimming of 
surface b rightn ess, discussed by Yoshii ( 1993 ) and Totani 
& Yoshii (2000), affecting the very faint part of the sample 
at the limit of the selection and then unable to invalidate 
our conclusions. 

The quantity Vmax used in the l/Vmax method to com- 
pute LFs can also be used to test the completeness of 
the sample: if the set of observed galaxies is complete, we 
expect that they populate uniformly the volume of the 
survey, i.e. that the galaxies are randomly distributed in- 
side their T^nax volume. This corresponds to the condition 
{V/Vraavi) = 0.5, whcrc V is the volume characteristic of 
each galaxy, given its redshift and the limiting magnitude 
of the survey. However, this line of reasoning is valid only 
if the population does not evolve in luminosity and it is 
spatially homogeneous. Larger or smaller values can have 
different origins. When the sample is subject to magni- 
tude incompleteness to the limiting magnitude (the more 
distant galaxies become undetectable), the volume Knax 



becomes too big and we have (F/l^iax) < 0.5. The same 
effect can be the result of luminosity evolution, if the near- 
est objects are also the intrinsically brightest ones. A value 
{V/Vraayi) > 0.5 could bc the effect of luminosity evolution, 
with the brightest objects being the most distant ones. In 
Sect. ^ we list the values of (V/Vmax) averaged over 100 
Monte Carlo realizations, which actually range between 
0.43 and 0.54, thus very close to the theoretical complete- 
ness value. 



4.5. Test of the method through mock catalogues 

The reliability of the method has been tested by means of 
mock catalogues. To this aim, we used template galaxies 
belonging to four spectral typ es, bu ilt from the GISSEL98 
library (Bruzual & Chariot 1993), corresponding to a 
star formation in a single burst, star formations with 
timescales r = 3Gyr and t ~ 15Gyr, and a continu- 
ous star formation, each one with Scalo ( |1986D IMF. The 
four spectral types match the coulours of Elliptical, Sa, 
Sc and Irregular galaxies. A single, nonevolving luminos- 
ity function is used, with a fraction of 0* assigned to each 
type following the mix of morphological types in the local 
universe used by Pozzetti et al. ( |1996D to build their PLE 
models. In particular, we assigned a fraction of 0.28, 0.47, 
0.22 and 0.03 to the four types, from early to late, assum- 
ing that these fractions remain valid beyond the original 
limit of bj < 16.5. 

The number of galaxies in a redshift slice [z, z + Az] 
and in a range of absolute magnitude [M, M + AM] has 
been computed by the following integrals: 



N{z, z + Az;M,M + AM) = 

pz+Az pM+AM 
Jz JM 



dV 

(l)inputiM)dM—dz, (9) 

dz 



where lo is the solid angle covered by the survey, (/)input (M) 
is the input Luminosity Function and dV is the volume, 
dependent on the cosmology. Apparent magnitudes in all 
the filters are computed from the absolute magnitude M, 
the template spectra and the redshift z. We decided to 
compute apparent magnitudes using the inversion of equa- 
tion H, viz applying only the fc-correction on the evolved 
SED, because here we are interested only in testing the 
reliability of the output compared with the input data, 
using the same procedure adopted for real data. In a real- 
istic case the evolutionary correction becomes important: 
recovering absolute magnitudes with equation ^ should 
produce an evolution of the measured LF even if the in- 
put LF is nonevolving, as discussed in Sect. 1.3 . 

For the input LF we imposed a Schechter functional 
form in the K band, with parameters a = —1.40, M^ = 
—23.14 in AB magnitudes and a normalization 0* = 
0.002 Mpc"^ setting Hq = 50kms"iMpc~^ We used the 
same cosmology to build mock catalogues and to recover 
the LF. We discuss the influence of the world models in 



this kind of calculation in Sect. 5.3 
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Fig. 10. The input and recovered LFs for 3 mock catalogues in two redshift ranges, selecting objects with K < 24, i.e. 
S/N > 3. Triangles and circles represent the binned LFs recovered with the l/Vmax and the C~ methods respectively. 
The points relative to the 1/I4nax method have been shifted by —0.15 magnitudes to avoid superpositions. Error 
bars represent 10 and 90% of the distribution of values obtained with the Monte Carlo method. The solid line is the 
Schechter function recovered using the STY method, and the dotted lines are the 10 and 90% STY functions. The 
thick dashed line is the input LF. 
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Fig. 11. Comparison between the input redshift ^modoi 
and the photometric redshift best fit Zphot for the objects 
with K < 24 

in the simulated catalogue used to recover the input LF in 
the upper panel of Figure Dotted lines identify the two 
considered redshift bins. 



In our simulated catalogues, we reproduce the same 
observational effects affecting the real catalogues of the 
HDFs. In particular, the signal-to-noise ratio behaviour 
in each filter band is set to be consistent with the data. 
Galaxies included in the mock catalogues have magni- 
tudes brighter than the limiting magnitude, otherwise 
their magnitude is set equal to 99, corresponding to a non 
detected object in the syntax of hyperz. The limiting mag- 
nitude corresponds to a signal-to-noise ratio S/N — 1, for 
consistency with the real HDFs catalogues, but we consid- 
ered only objects with S/N > 3 to estimate LFs. The pho- 
tometric error is computed as a function of magnitude for 
each filter, after specifying a signal-to-noise ratio reached 
at a given magnitude, matching the same ratios computed 
for the HDFs catalogues and using a similar procedure as 
in Bolzonella et al. (200C). A random reddening with Ay 



ranging from to 1 is also applied to SEDs. To allow a pho- 
tometric redshift estimate, we included in the catalogues 
only objects detected in at least 3 filters. The number of 
galaxies included in a mock catalogue has been computed 
using a surface similar to the HDFs, i.e. Sarcmin^. In this 
way we obtained A'obj objects, to which we added a fluc- 
tuation due to Poissonian statistics, ±random[0, y^^obj]. 
Objects have been randomly selected from a larger field, 
to allow the selection of bright objects at low redshift. The 
final catalogues have roughly the same number of objects 
as the HDFs, with the same observational characteristics. 

Next, we computed photometric redshifts for these 
simulated galaxies, using all the available SEDs, and we 



used the hyperz outputs (the probability function of red- 
shift, the SED parameters) to estimate absolute magni- 
tudes and then the Luminosity Functions by means of the 
Monte Carlo method described in the previous section, se- 
lecting objects in the Kg and with magnitudes Kg < 24, 
i.e. S/N > 3. 

The recovered LFs for 3 random catalogues are shown 
in Fig. |l^ for two redshift ranges. The mean parameters 
obtained averaging over a set of 10 random catalogues 
are ^ = 0.00267 ± 0.00141 Mpc"^, = -22.958 ± 

0.771 and a = -1.441 ± 0.083 in the redshift range z = 
- 1. The estimated errors in these values correspond to 
the standard deviation (Icr) of the distribution of values 
used to compute the arithmetic mean and do not take 
into account the errors inferred from each Monte Carlo 
realization. The agreement of these values with the input 
ones is remarkable. In the redshift range z = 1 - 2 the 
recovered parameters are 0* = 0.00259 ± 0.00207 Mpc"^, 
= -22.611 ± 0.489, a = -1.681 ± 0.208. In this case 
the normalization is well recovered, even if the large error 
refiects the large scatter in the values obtained in different 
realizations; the value of is consistent with the input 
one, whereas the recovered a is slightly overestimated even 
considering the la error. 

The comparison between the input value of the red- 
shift and the photometric redshift best fit is shown in 
Fig. |ll| for the objects with S/N > 3 in K magnitudes: 
for these objects the photometric redshift is a very good 
estimate of the input one and we do not see the degen- 
eracy between different redshift ranges, characteristic of 
the faintest objects with lower signal-to-noise ratio (see 
Bolzonella et al. 2000). Only few objects are erroneously 
attributed to high redshifts: these objects are very faint 
galaxies, non detected in the U^qq and V450 filters, deter- 
mining a confusion in the location of the Lyman break. 

In Fig. ^ the input absolute magnitude is compared 
to the absolute magnitude computed by hyperz, using the 
redshift and the spectral type best fit. The objects repre- 
sented in these two panels have been selected using their 
photometric and their input redshifts, in the two redshift 
bins shown in Fig. We considered only objects with 
K < 24, i.e. the same objects used in the LF estimate. 
The agreement is very good, with very few outliers. In 
the left panel, we can notice the rapid decrease of galaxies 
with Mk —16, producing large error bars in the binned 
estimate of the LF. In the range between z — 1 and 2, the 
lack of faint objects due to observational limits prevents 
a good estimate of the faint-end slope a using the STY 
method. Nevertheless, the bright end is well reproduced 
by the binned methods. 

Similar analysis comparing methods for LF estimate 
through simulations have been carried out by other au- 
thors. Willmer ( |1997D found that the STY method tends 
to slightly underestimate the faint-end compared to the 
input value. In our case, we found a small overestimate of 
a, but the large error bars are consistent with the input 
value. Concerning the non parametric methods, the study 



of Takeuchi et al. (2000) demonstrated that for large and 
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Fig. 12. Comparison between the input absolute magnitude Mk, model and the absolute magnitude computed using 
the photometric redshift best fit AfK.phot, for the mock catalogue used in the upper panel of Fig. and objects with 
K < 24:. Left: redshift range z = - 1. Right: z = 1-2. Circles and crosses represent the points selected following the 
input redshift and the photometric redshift respectively. 



spatially homogeneous samples the LF estimate is not bi- 
ased, whereas the faint-end is subject to large fluctuations 
when the sample is small. The overestimate of the low red- 
shift LF on the faintest bins is similar to ours, according 
to their figures, and still consistent with the input LF due 
to the large error bars. On the contrary, at higher red- 
shift we see a decline in th e faintest bins, that is also been 
shown by Liu et al. ( 199^ ). 

In summary, the procedure used to recover the LF in 
the range z = - 1 can be considered as reliable, and 
we did not try to take into account the small systematic 
effects mentioned above. In the range z = 1 - 2, the results 
of the STY method have to be taken with care, but the 
non parametric estimate of the LF still provides a good 
fit of the bright-end. 

5. Near infrared luminosity functions in the HDFs 

We co mput e the LFs using the three methods described 
in Sect. 4.1, In particular, for the non-parametric methods 
we adopt a binning of 1 magnitude or 2 magnitudes at the 
faint end to have a conspicuous number of objects in each 
bin. To compute the luminosity functions we divide the 
sample in redshift slices, larger than the typical errors of 
photometric redshifts, to minimize the change of redshift 
bin and to study the redshift evolution of the LFs. The 
adopted limiting magnitude is Kg = 24 in both the HDF- 
N and HDF-S. Following the procedure described above, 
we iterate the computation of the binned or parametrized 
LF. We choose to realize 100 iterations, sufficient to es- 
timate the effect of random selection of redshifts. During 



the photometric redshift calculation we impose a range 
of absolute B magnitudes: solutions with Mb outside the 
range [—28, —9] are considered forbidden also when esti- 
mating the LF. 

Because the i^Tg-band is the reddest one, the abso- 
lute magnitudes have been computed using always the 
apparent magnitudes. Studying the K-hand at redshifts 
z ~ 2 means to map the rest-frame /-band emission: in 
Fig. we show that these magnitudes are strictly corre- 
lated and thus they basically map the same stellar pop- 
ulation. This behaviour can be explained considering the 
emitting stellar population: even at z 2 the luminosity 
at the wavelengths covered by the Kg filter is always pro- 
duced by the old star population, the 4000 A break still be- 
ing inside the J filter. Furthermore, Kg magnitudes are not 
affected by recent bursts of star formation. For this rea- 
son, and because o f th e characteristics of the fc-correction 
discussed in Sect. L3, we considered that we can safely 
compute i^s-band absolute magnitudes at least up to a 
redshift of 2. 

We have also estimated the J-band LF in the redshift 
range z = [0, 1] from the J-selected subsample, and in the 
redshift range [1, 2] by selecting the objects in the i^s-band 
sample that better approximate the J filter rest frame. 

We will study the LF in the optical bands and in the 
UV in a forthcoming paper (Bolzonella et al. in prepara- 
tion). 
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Fig. 13. NIR colour-colour plot: /814 — Kg vs /814 — J for the HDF-N and HDF-S. Limiting magnitudes are /814 = 28.5, 
J = 24.6 and Ks = 24.0, corresponding to S/N = 3. 



5.1. Ks-band LF 

Up to now, the i^Tg-band LF has been estimated only for 
data in redshift surveys, and for this reason the accessible 
range of redshifts to be explored was hmited. In Table | 
we report previous estimates, mainly retrieved from the 
table by Loveday ( 2000 ), completed with a guess of the 
magnitude and redshift ranges of the samples. Magnitudes 
have been transformed into the AB system by means of 
the conversion provided in Table |^ and the dependence 
from the Hubble constant has been explicited using the 
Hubble parameter h = 7Jo/(100kms^^ Mpc""'^). 

Using the HDF-N and HDF-S catalogues we can reach 
unprecedented depths: we estimated for the first time the 
LFs in the Kg band for objects with Zphot G [0, 1] and 
[1,2]. 

We selected the subsamples in each redshift range after 
the randomization procedure of redshifts. The values of 
(y/Vniax) averaged over the Monte Carlo realizations are 
0.54, 0.44 in the redshift ranges Zphot e [0, 1], [1, 2] for the 
HDF-N and 0.51, 0.47 for the HDF-S in the same redshift 
ranges. 



Figure 14 illustrates our estimate of the Kg band LFs 
for the HDF-N and the HDF-S. There is a good agreement 
between the local 7^-band LF computed by Cowie et al. 



(1994, 199(:) and the Zphot G [0, 1] sample, especially in the 
HDF-S. In the case of the Zphot G [1, 2] sample, our results 
are still compatible with the local values. A slight nega- 
tive evolution is observed between the [0, 1] and [1, 2] bins 
when comparing the non-parametric estimates for galaxies 
fainter than Mk = — 21. This trend is hardly significant. 
The fact that the l/Vmax and C~ estimates are very close 
means that there are no artifacts due to clustering. 



In Table ^ we list the parameters of the STY estimate 
for the [0, 1] and [1, 2] samples. The most impressive re- 
sults we can note from Fig. |lj are the very wide range 
of absolute magnitudes covered by the data and the pos- 
sibility of computing for the first time the NIR LFs at 
redshifts in the range [1,2], where many difficulties arise 
for the traditional spectroscopy. Moreover, this redshift 
range is of paramount importance in the study of galaxy 
formation and evolution, as we will discuss in Sect. ^. 

In Fig. |l^ we summarize the results given in Table | 
showing the a and M^(AB) parameters and their respec- 
tive errors as a function of redshift, derived for the two 
adopted cosmologies. 



5.2. J-band LF 

We also computed LFs in the J-band. In this case, we se- 
lected galaxies in the J filter when considering the lowest 
redshift bin [0,1], whereas we estimated the J-band in the 
highest redshift range (zphot G [li2]) using the A's-band 
selected subsamples. In this way we select the objects ap- 
proximately in the J-band rest-frame and we can check if 
the assumptions made for the Kg band LF computation 
were safe. We selected objects in the HDF-N with J < 24.6 
in the redshift range [0, 1] and with Kg < 24 in z = [1,2], 
corresponding to objects with S/N > 3. At these limits 
the colours in the HDF-N and HDF-S arc very similar: at 
J = 24.6, the mean /gu - J in [0, 1] is 0.57 in the HDF-N 
and 0.50 in the HDF-S; at Kg = 24 the mean Isu - K 
in z = [1,2] is 1.65 in the HDF-N and it is 1.85 in the 
HDF-S. 
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Table 2. Compilation of other i^s-band LFs for field galaxies in redshift surveys, in general obtained with cosmological 
parameters fig = 1; the data are transformed into AB magnitudes in the filter K. Errors are in general given at la of 
confidence likelihood contours. Modified and updated from Loveday's (2000) table. 



Author(s) 





— 5 log h 


a 


Mk — 5 lo 


T li range 


z range 


other remarks 


0.0112±0.0016 


— 21.51±0.3^ 


— 1.0±0.3 


[—23.5, 


— 19.0] 




7 < n 1 


AARS 


0.029±0.007 


-21.15±0.23^ 


-1.04±0.31 


[-22.5, 


-18.5] 


0.0 < 


z < 0.2 




0.019±0.002 


-21.44±0.11^ 


-1.04±0.31 


[-24.0, 


-18.5] 


0.0 < 


z < 0.8 




0.016 


-21.63 


-1.25 


[-23.5, 


-16.5] 


0.0 < 


z <, 1.6 


Jf-band selected 


0.0182 


-21.50±0.17 


-1.03±0.24 


[-23.5, 


-18.0] 


{z)-- 


= 0.14 


if-band selected 


0.012±0.004 


-21.72±0.3 


-1.27±0.2 


[-23.5, 


-18.5] 


0.0 < 


z <, 0.5 


if-band selected 


0.012±0.008 


-21.71±0.42 


-1.16±0.19 


[-24.0, 


-14.0] 


{z)-- 


= 0.05 


Stromlo-APM 


0.0116±0.001 


-21.52±0.05 


-1.09±0.06 


[-24.0, 


-18.5] 


0.01 ^ 


z <, 0.05 


2MASS 


0.0108±0.0016 


"21.57±0.03 


-0.96±0.05 


[-24.0, 


-17.0] 


< . 


2 ^ 0.2 


2dF - ACDM 




-21.61±0.08 


-1.10±0.14 


[-24.0, 


-18.0] 


Q<z 


^ 0.18 





Mobasher et al. (|l993ir 
Glazebrook et al. (1995) 
Glazebrook e t al. (1995 ) 
Cowie et al. (|l996|) 



Gardner et al. (\L99W 
Szokoly et al. (|L998^ 



Loveday ( |2000| ) 
Kochanek etaL (|00l|)* 
Cole et al. (l200l|) 



Balogh et al. (2001) 



^ From Loveday (2000): added 0.22 magnitudes due to different method for fc-corrections. 

^ From Loveday ( 2000 ): aperture correction of —0.30 magnitudes. 

^ The value adopted is relative to the go ~ 0.5 model with only fc-correction. 

* Isophotal magnitude selection 7 < K20 < 11.25. 



Normalized to the weighted number of galaxies brighter than /Cvcg 



-21.5. 



Table 3. Parameters of the Ks-hand LF for galaxies in the HDF-N and HDF-S obtained with the STY method. 



2 range 


Cosmology 


Field 


0* [h^Mpc-^] 


M^(AB) - 5 log /i 


a 


Mk - 5 lo 


g h range 


0<z<l 


Qo — 1, f^A ~ 







HDF-N 




-21.540l°-i^^ 


l.lUi_0 070 


[-23.0, 


-12.0] 










HDF-S 




-21.860+0 2TO 


-1 159+°093 


[-23.0, 


-12.0] 




fio = 0.3, Qa 




0.7 


HDF-N 


U.U±Z±_0 003g 


-22.2661°:^^? 


_1 154+0.064 
1.1D'±„0 07g 


[-24.0, 


-12.0] 










HDF-S 




—22 S8Q+°-^^'' 


1 17n+0-089 
l.i (U_0 0(j5 


[-23.0, 


-12.0] 


1< z <2 


0,0 — 1, Qa ~ 







HDF-N 


0.0050l°»»f7 


-22.524l°;33^^ 


1 c;7Q+0.098 
1.0 { »_0.o7g 


[-23.5, 


-17.0] 










HDF-S 


0.0083t°:S«S 


-22.2011?:°^ 


_1 411+0.080 


[-22.5, 


-17.5] 




fio = 0.3, Qa 




0.7 


HDF-N 


0017+°"°"** 

U.UUl/_0 0005 


-23.200l°:f7° 


1 c,KS+0099 
l.JOO_0 0gg 


[-24.0, 


-17.5] 










HDF-S 


0024+° °°^'' 

U.UUZ^_0 0017 




J^-4^4_0.174 


[-23.0, 


-18.0] 



Table 4. Parameters of the J-band LF for galaxies in the HDF-N and HDF-S obtained with the STY method. 



z range 


Cosmology 


Field 


(j}* [h^Mpc-^] 


M}(AB) - 5 log ft 


a 


Mj — 5 log h range 


< z < 1 


0,0 = 1, Oa = 





HDF-N 


0357+" °°°'' 

u.uoor_0 g0g4 


91 cq7+0.225 
Zl.aol _o 264 


-1 106+°°^^ 

i-iUD_0 055 


[-23.0,-11.0] 








HDF-S 


n nosi +0.0086 

U.U^»i_0 0049 


-21.942l°-«° 


-1 090+°°®^ 

1. U»U_0 046 


[-23.0,-11.0] 




fio = 0.3, flA 


= 0.7 


HDF-N 


U.U1O5_0 0027 


-22.077l°:Jg 


-l-066lHi^ 


[-24.0, -11.5] 








HDF-S 


0.0129lg:«f5 


-22 451 +0-244 


-1.082t°:°I? 


[-23.5,-11.5] 


1 < 2 < 2 


Slo = 1, SIa = 





HDF-N 




-21.788l°;423 


-1.080l°:lf5 


[-23.5,-17.0] 








HDF-S 


a 0251+° °°** 

U.UZO1_0 0j^26 


-21 439+0-400 


770+0.199 

' ' 'J-0.301 


[-22.0,-17.0] 




fio = 0.3, flA 


= 0.7 


HDF-N 


n 0061+°°°^* 

U.UUDi_0 0036 


-22 4S1+0-376 
^^•^■J -"--1.652 


l.Uy:r_0 jg5 


[-24.0,-17.5] 








HDF-S 


0074+° °°2® 
u.uury:_0 0040 


99 QP;7+0.605 
zz.ou 1 _i 273 


-0 796+0-229 

U. (»O_0 3j^7 


[-23.0,-17.5] 



The values of (T^/V,nax) are 0.56,0.43 in the redshift 
ranges ^phot £ [0, 1] and [1,2], respectively, for the HDF-N, 
and 0.51,0.47 in the same redshift ranges for the HDF-S. 

In Fig. |l^ we plot the LFs obtained with the adopted 
parametric and non parametric methods in the redshift 



ranges [0, 1] and [1, 2], as well as the Cole et al. (2001) local 
LF estimated in the 2dFGRS, shown as a reference. Our 
estimate and the Cole et al. ( pOOl ) one, suitably trans- 



formed in AB magnitudes {M} = -21.40, a ^ -0.93, 
(f)* = 0.0108 computed with cosmology 17o = f,i^A = 
and only fc-correction to match the same conditions we 
used), seem to be in disagreement, mainly in the normal- 
ization. However, the comparison between our LF estimate 
obtained in the flat A-dominated cosmo logy and the anal- 
ogous one computed by Cole et al. 2001 partially mitigates 
the difference. 
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Fig. 14. Xs-band LFs for galaxies in the HDF-N and HDF-S in two redshift bins, assuming a limiting magnitude 
Ks = 24.0 and a cosmological model with ilo = 1. Triangles and circles represent the median of the results obtained 
with the 1/Vmax and the methods respectively. The upper panels represent the LF for galaxies with Zphot in [0, 1], 
the lower in [1,2]. The points relative to the method have been shifted of —0.15 magnitudes for clarity. The 

error bars corresponding to 10 and 90 percentiles of the distribution are also shown. The solid line shows the LF 
obtained with the STY method. The LF obtained by Cowie et al. ( 1996| ) (dashed line) is shown as reference LF. In 
the lower panels, the Zp^ot — [0, 1] LF (thin solid line) is also displayed for comparison. 



Table U contains the values of the Schechter parameters 
of the J-band LF obtained in the two redshift ranges for 
the HDF-N and HDF-S. We have also computed the LF in 
the H and / bands for the HDF-N and HDF-S catalogues. 
In all cases, we found similar results for the two fields. 



5.3. Effect of cosmology 

We estimated LFs in two cosmologies: the "old" Standard 
CDM with r^o = 1 a-nd J^a = 0, and a flat cosmological 
constant dominated model, that nowadays is the most ac- 
cepted one, with Q,q ~ 0.3 and Q.a — 0.7. The influence 
of cosmology in the photometric redshift computation is 
negligible, as discussed by Bolzonella et al. (200C), 



thus 

we used the same photometric redshifts to estimate the 
LFs in different cosmologies. 



When LFs are computed at low redshifts, we expect 
little or no differences in the estimates, as in the case of 



the 2dFGRS by Cole et al. (|200l| ), where the difference 



in the value of Af^ between the two interesting mod- 
els is 0.16 magnitudes. Therefore, whereas local estimates 
are not affected by the change of cosmology, when we are 
dealing in particular with galaxies at high redshifts the 
variation of volume and distances become substantial. As 
expected, looking at Tables || and || we can remark that 
the value of M* brightens when the lambda dominated 
cosmology is assumed, whereas <j)* decreases because of 
the increase of the comoving volume at a given redshift. 
In fact, the difference in distance modulus will affect abso- 
lute magnitudes, whereas the difference in volumes affects 
the estimate of the cf)* parameter. 
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Fig. 15. Parameters of the LFs as a function of the redshift bin for the two cosmologies adopted in the paper. Results 
obtained with the Hq = 1 and fio = 0.3, flA = 0.7 cosmologies have been shifted in redshift by —0.05 and +0.05 
respectively. 
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Fig. 16. J-band LFs for galaxies in the HDF-N and HDF-S in two redshift bins, assuming a limiting magnitude 
J — 24.6 in the redshift range [0, 1] and Kg = 24.0 in z — [1,2], and a cosmological model with f^o = 1. Same symbols 
as in Fig. |lj. The LF obtained by Cole et al. ( 200l|) (dashed line) assuming the same cosmological model is also shown 
as reference LF. In the lower panels, the LF in the Zphot — [0, 1] interval is also displayed for comparison (thin solid 
line) . 
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Fig. 17. The luminosity density in the A'g-band, as obtained from the values listed in Table g. Squares: luminosity 
densities computed using the median values of the STY estimate of the LF in the case of a fio = 1 cosmological model. 
Circles: values of the luminosity density in a flat cosmological constant dominated universe. The value obtained by 
Cole et al. (|200l|) is also shown for comparison as a triangle. 



5.4. NIR Luminosity Density 

Using the values of the Schechter parameters listed in 
Tables ^ and ^ we computed the luminosity density, given 

by 



PL= J L(I){L) dL = (t>*L*r{a + 2) 



(10) 



Results are shown in Fig. |T^, in units of /iWMpc"^, 
derived from the usual solar luminosity values, after in- 
tegration and scaling through a stellar solar type SED 
(taken from the library of Pickles 1998). Boxes represent 
the maximum and minimum values obtained using the 
Schechter parameters in a given redshift range. We show 
the results for both the adopted cosmological models: we 
can see that the luminosity density does not strongly de- 
pend on cosmology, being the values relative to one model 
inside the boxes of the second one. The most remarkable 
characteristic visible in Fig. |l^ is that the luminosity den- 
sity is consistent with non-evolution at least up to a red- 
shift 2-2. 

As expected from the comparison of our Schechter pa- 
rameters with the values in Table ^ our estimates of 
are higher than the luminosity densities computed for in- 
stance by Cole et al. (^OOlD for the 2dFGRS: their value in 



the /fs-band, conveniently converted into /iWMpc in 
the Ks filter, is pK, = 3.5 x lO^^hWUpc'^. It is shown 
as a triangle in Fig. ^: it is marginally consistent with 
our lower limit of the z £ [0, 1] estimate in the HDF-N 
and HDF-S for the A-dominated model (the same model 
adopted by Cole et al. 2001). In the J-band, our estimates 
in the redshift range [0, 1] are pj = 3.53 x 10^4(3.86 x lO^*) 
and 2.35 x 10^4(2.49 x lO^^) /i WMpc^^ in the HDF-N 
(HDF-S), for the matter and A-dominated models respec- 
tively, whereas the value found by Cole et al. (2001) is 
6.8 X 1033/iWMpc-3. 



In a recent paper, Wright ( ^001) alre ady claimed that 
the value obtained by Cole et al. ( 2001 ) does not match 
the extrapolation to the NIR from the optical luminosity 
densities obtained in the SDSS by Blanton et al. ( 2001 ). 
The discrepancy is about a factor of 2.3. On the contrary, 
the present results are in much better agreement with the 
SDSS optical luminosity densities. This result has to be 
considered with caution because of the small size of the 
HDFs. 

5.5. Mass Function 

The X-band luminosity is considered as a good tracer 
of the stellar mass in galaxies, because the bulk of the 
emission at these wavelengths is dominated by solar stars, 
that constitute most of the mass in stars. Moreover, the 
K-hand estimator is nearly independent on the star for- 
mation history and then it can be safely used to compute 
the baryonic mass contained in galaxies. 

We adopted a constant stellar mass-to-light ratio: 
Msta-r/LK — 0-S> Mq/ Lkq , b. m ean value obtained from 
the estimate by Worthey (1994) considering the range of 
ages spanned by our objects. In any case the dependence 
of M/L with age is not strong in this filter. With this as- 
sumption, the LF reflects the stellar mass function: a non 
evolution in luminosities implies a non evolution in charac- 
teristic masses. In Fig. ^ we show the masses computed 
from the i^j-band apparent magnitudes of the Kg-hand 
selected subsample used in this study. We also show the 
limiting mass as a function of redshift computed for an 
elliptical galaxy, assuming Kg-hand limiting magnitudes 
of 24.0 at which reach S/N = 3. The change using differ- 
ent SEDs is negligible. According to this figure, the HDFs 
allow to detect stellar haloes of ^ IQ^Mq up to z — 1, 
and around 1O^°M0 up to z ^ 3. 
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Fig. 18. The masses of the objects in the Kg-band selected subsamples with Kg < 24.0 in the HDF-N and HDF-S, 
computed assuming a mass-to-light ration of 0.8 in solar units. The solid line represents the limiting mass inferred 
from the limiting magnitude Kg = 24.0. 



6. Discussion: Galaxy formation models 

LFs are a convenient way to describe the galaxy popula- 
tion and to get hints about the mechanisms of formation 
and evolution of galaxies. 

Two scenarios are in competition to explain the his- 
tory of galaxies up to the present epoch. The formation of 
elliptical galaxies is especially intriguing, because despite 
their old and apparently simple stellar population, their 
process of formation is far from being understood. The 
two models in competition are: 

— The hierarchical scenario, which is based on CDM cos- 



mological models (e.g. Kauffmann et al. 1994; Cole et 
al. 2000). It assumes that galaxies have formed through 
merging of sub-galactic structures. In particular, ellip- 
tical galaxies are born from the merging of two disks of 
comparable dimensions; during the interaction, a burst 
of star formation occurs, consuming the gas present in 
the progenitors, and then passive evolution follows. 
The monolithic scenario, revisited in different ways by 
many authors, which basically assumes that galaxies 
formed approximately at the same epoch, with partic- 
ular details depending on the morphological types (e.g. 
Rocca-Volmerange 



Guiderdoni 1991 



Pozzetti et al. 
1996). Ellipticals are believed to form at Zform ^2 — 3 



and, at the time of their assembly, a burst of star for- 
mation occurred, followed by passive evolution of the 
stellar population. 

The two scenarios foresee different characteristics as 
a function of redshift for the progenitors of the blue and 
red galaxies in the local universe. Moreover, since most of 
the merging would take place at z < 2, the redshift range 
between 1 and 2 is fundamental to discriminate between 
monolithic and hierarchical scenarios. Both mechanisms 
account for the properties of elliptical galaxies up to z ~ 1, 
e.g. the C M relat ion (Gladders et al. |1998 ; 
& Chariot |1998bD, but, between z 



expectations for the photometric properties of all galaxies 
differ significantly between the two scenarios. 

A powerful test to establish what drives the galaxy evo- 
lution, was proposed by Kauffmann & Chariot ( 1998| ), us- 
ing the X-band luminosity function. The luminosity in the 
K filter is directly linked to the mass in stars and is barely 
affected by the presence of dust extinction, thus making 
the K photometry a privileged tool to study galaxy forma- 
tion. Moreover, galaxies with the same stellar mass have 
nearly the same K magnitude, independently on their star 
formation history. For these reasons the if-band LF can 
probe if galaxies were assembled early, according to the 
monolithic scenario, or recently from mergers. Kauffmann 
& Chariot (1998) built two PLE models with density pa- 
rameters rio = 1 and r^o = 0.2 and two hierarchical mod- 
els based on the CDM cosmology, with the same values of 
r^o- They computed the evolution of the if-band LF at in- 
creasing redshifts for the PLE models and for the Oq = 1 
hierarchical model, each one able to reproduce the local 
LF. On the contrary, their low density hierarchical model 
failed to reproduce the local K-hand LF and they did not 
compute the evolution of the LF in this framework. 

At redshifts z > 1 a sharp difference between the two 
models is predicted, as explained in Kauffmann & Chariot. 
The PLE models foresee a constant bright-end for the 
LF, with small differences between the flat and the open 
cosmology, whereas the hierarchical model considered by 
Kauffmann & Chariot undergoes a shift toward faint mag- 
nitudes (see Fig. [l9| ). 

The development of hierarchical scenarios in open or 
flat cosmological models with cosmological constant miti- 
gates the discrepancy between these two scenarios (mono- 
lithic vs. hierarchical), because the epoch of the major 
merging moves at higher redshifts (Fontana et al 



1 and 



Kauffmann 
z = 2, the 



1999 



Cole et al. 200C). In this case, the PLE model could not 
be distinguished from a scenario where the galaxy assem- 
bly occurs at early epochs, followed by a passive evolution 
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Fig. 19. Comparison between the theoretical luminosity 
functions derived by Kauffmann & Chariot (1998) and 
the present results on the HDFs. Histograms represent 
the prediction for the hierarchical, fio = 1 CDM, (dashed 
line) and PLE scenarios (solid line). The solid and long 
dashed continuous lines display respectively the STY LFs 
fitted to the data in the HDF-N and HDF-S. Circles and 
triangles show the estimate of the LF with the method 
in the HDF-N and HDF-S. In the lower panel we show the 
model predictions both at z = 1 and z = 2, because we 
estimated the LF using galaxies with redshifts between 
these two extremes. 



of the stellar population. Our results for the cumulative 
redshift distribution actually support such scenarios (see 
Fig.|). 

The present results, i.e. the lack of significant evolu- 
tion in the bright part of the LF from the redshift range 
[0, 1] to [1, 2], provide a stringent clue, supporting the idea 
that massive galaxies were already in place at high red- 
shifts, against the old CDM hierarchical model adopted 
by Kauffmann & Chariot. The comparison between these 
theoretical predictions and the observations derived in the 
present paper can be found in Fig. In the lower redshift 
bin, our estimates of the LFs present a faint-end slope in 
agreement with the hierarchical model, whereas at bright 
magnitudes the small differences between the two models 
do not support any claim. In the redshift bin z ~ [1,2] 
we have compared our estimate with both the predictions 



of the models at redshift z — 1 and 2: the model predic- 
tions suitable for our sample should lie between the two. 
It is evident from the lower panel of Fig. |l^ that the very 
bright part of the LFs remains well above the hierarchical 
model predictions at z = 1 and 2, being much closer to 
the predictions of the monolithic/PLE-like scenario. On 
the other hand, the faint end slope seems to be in bet- 
ter agreement with the hierarchical model, even in this 
redshift range. 

Other recent studies on the Hubble fields, based on 
different selection criteria, seem to indicate that the for- 
mation of elliptical galaxies should be placed at z > 2 
(Bemtez et al. 1999| ; Broadhurst & Bouwens 2000). In the 
present paper we do not select galaxies according to mor- 
phological types, but the same conclusions apply to the 
most massive (NIR luminous) galaxies in our sample. 

The lack of evolution in the bright end of the LF is in 
good agreement with the results found from spectroscopic 
surveys. Glazebrook et al. ( 1995| ) found no evidence for 
evolution in the K-hand LF up to z < 0.5, and concluded 
that massive spheroids were in pl ace at z > 1 and then 
evolved passively. Songaila et al. ( 1994 ) found a lack of 
significant evolution in the ir K- hand sample up to a red- 
shift of '--^ 1. Cowie et al. ( 1996 ) found little evolution in 
their sample of red (old) objects to z ~ 1. The present re- 
sults extend the previous findings in redshift, up to z ~ 2, 
with the same conclusions with respect to the evolution of 
the most massive galaxies. The comparison between the 
present LFs in the near-IR and in the optical-UV bands 
(Bolzonella et al. in preparation) will provide new insights 
on the galaxy formation scenario. 



7. Summary 

The results of this paper can be summarized as follows: 

1. The photometric redshift technique is an indispensable 
tool to study the faint-end and the evolution of LFs, 
pushing the limiting magnitudes toward fainter limits 
than the works based on spectroscopic surveys. 

2. We have elaborated a method to compute the luminos- 
ity function properly taking into account the charac- 
teristic of the uncertainties of the photometric redshift 
technique. This approach can be conveniently applied 
to compute LFs from the forthcoming wide and deep 
field photometric surveys. Moreover, this method can 
be easily extended to other wavelengths crucial for the 
study of galaxy evolution, as optical LFs and the LF 
at 1500 A (Bolzonella et al. in preparation). 

3. We have computed the LF in Kg and J bands 
for the HDF-N and HDF-S catalogues with two 
non-parametric methods, 1/Vinax and C~ , and with 
the maximum likelihood method STY, assuming a 
Schechter functional shape. We found similar results 
for the two fields. This seems to indicate that clus- 
tering does not considerably affect the estimate of the 
LFs, even for the l/Knax method. 
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The bright-end of the NIR LF remains unchanged 
within the errors up to at least z ~ 2. Also the 
NIR luminosity density is in good agreement with no- 
evolution, at least up to a redshift 2; ~ 2. This result 
is consistent with the bulk of the stellar mass being in 
place and assembled at such redshift. 
The analysis of the near-IR LF and the cumulative red- 
shift distributions is a powerful test for galaxy forma- 
tion models. The evidence of an unevolving bright-end 
of the iiTs-band LF up to z ~ 2 supports the assembly 
of massive galaxies before such redshift, a result which 
is rather close to the old monolithic scenario than to 
the hierarchical standard CDM cosmological model. 
Our results are consistent with hierarchical scenarios 
in open or flat cosmological models with cosmological 
constant (such as Fontana et al. 1999 Cole et al. 2000| ). 
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